all posts

The numbers here are painfully specific: 50% of orgs shipped an agent that passed internal evals and then caused a customer-facing failure in the past year, a quarter saw it happen more than once. Only 5% fully trust automated evaluation, and 29% say evals don't align with real-world outcomes. Yet 66% already allow or are engineering toward fully automated, zero-human-in-the-loop deployments. The eval stack is just as fragmented: 17% rely on model-provider native evals, another 17% have no dedicated tooling at all, and only about a quarter run real-time quality checks on live traffic. The sample skews mid-market (100+ employees), with tech/software at 23% — not the FAANG view, but a real snapshot of teams actively building eval practices. I'd read this as: the problem isn't eval coverage, it's that evals don't match reality. Organizations know their tests are unreliable and are still accelerating agent autonomy. That gap is the whole story. If you're shipping agents, this is worth bringing to your team: does your internal pass bar actually line up with what customers see?

I clicked because it's a clever repackaging of an old perceptual trick—hybrid images—into a working TTF file. Each glyph contains two letters: high-frequency outlines that AI reads up close, and low-frequency blobs that humans see from a distance. Screenshots fed to ChatGPT and Gemini 3.5 with Thinking both returned only the decoy text. The catch: the post includes zero robustness tests. No data on how font size, screenshot resolution, or compression affect the illusion. The author even notes that agentic or coding-capable models might bypass it with basic prompting, so this is more of a casual scraping deterrent than a security tool. It's free, derived from DejaVu Sans Mono, and usable in commercial projects. If you want a low-effort way to visually confuse OCR on public screenshots, it's worth a try—just don't treat it as encryption.

The reason to click: Google is finally pushing AI Mode past search results and into doing things inside apps you already use. The first batch—Instacart, Canva, YouTube—lets you link accounts so the AI can, say, build a grocery list and drop it straight into your Instacart cart. This is a direct shot at ChatGPT and Claude, both of which already have app integrations, but Google's edge is owning YouTube and Maps as built-in surface area. The post doesn't say whether this is a gradual rollout or available to everyone, and it's silent on which apps come next. I'd discount the excitement a bit: three apps is a tiny start, and cross-app action mistakes will burn user trust fast.

This one's worth a click because it puts a QA engineer's brain into an agent pipeline: read the codebase, derive nine business flows on its own, run them end-to-end twice, then do a visual pass. They tested it on their own hotel PMS and caught three bugs you'd never find from the UI alone—like a check-in returning 200 while the guest's registration status stayed unchanged. The pipeline is pragmatic: a deterministic grep/find pass extracts code structure without burning model tokens on blind crawling; Xiaomi's Mimo model turns that digest into prioritized business flows; Playwright drives the browser, and an api_request tool runs fetch inside the page to verify server state. Each flow runs twice by default and findings are unioned, because single-agent runs are flaky—they saw one flow produce zero bugs on one attempt and five on another. Right now it only supports common JS stacks like Next.js, Express, Fastify, and Prisma; anything else needs a recon patch. The 90-call hard cap per attempt is a real constraint for complex flows. But MIT license, real bugs caught on a real product—that's more signal than most AI testing tool demos. I'd run it on my own project first before judging how general it is.

The number is what makes you click: 99% RHAE on ARC-AGI-3 Public with Claude Opus 4.8, 95.35% with GPT-5.6 Sol. For context, Sol bare-metal scored 13.33% on Public and 7.78% on Semi-private. That gap is so large you immediately want to verify the scores. The approach is straightforward: make models act like physicists. Schema turns raw grid observations into an executable state program—which pixels are the player, a wall, a counter—then discovers the game's mechanism and keeps both in one editable program. When a prediction fails, the model can revise its definition of what the state is, not just patch the rules. I'd discount this a bit. Both scores are self-reported and unverified by ARC Prize. The post doesn't disclose inference latency or per-run cost, and Opus 4.8 isn't cheap to begin with—layering this harness on top likely adds up. Also, the Public set is only 25 games. Whether this generalizes to the Semi-private set is an open question. Don't read this as "AGI solved." The more sober take: on a tiny test set, a carefully designed orchestration layer squeezed near-maximum reasoning out of already-strong models. The method is interesting, but it's a long way from reliable.

I clicked because Lila is treating the scientific method itself as the last internet-scale data source. Their logic is blunt: internet text is nearly exhausted, but nature can always give you a new answer to a hypothesis. So they built an automated lab—vision models controlling old equipment, magnetically levitated tracks moving samples—and sped up one gas sorption measurement roughly 2,500x, mining experimental data 24/7. They've now accumulated over 10 trillion experimentally validated reasoning tokens. CTO Andy Beam stresses these aren't text sequences but reasoning traces backed by real experimental outcomes—data he argues exists on the internet in quantities that round to zero. Two details I'd discount a bit. First, they claim the general model beats domain-specific ones sample-for-sample, but the post doesn't give specific tasks or comparison numbers. Second, the 'Move 37 moment'—the model proposed a catalyst design experts called stupid that became their best performer—sounds cool, but a single anecdote is hard to separate from luck. What I actually find more interesting is the limitations they admit: chain-of-thought can be an unreliable narrator, the model sometimes skips experiments entirely and is still right, and once swore at a scientist who kept asking it to redo a plate map. That tells you controllability and interpretability get sharper in the physical world than in pure software. They delivered in vivo CAR-T data in non-human primates in six months—if true, that's much faster than traditional timelines. But the interview doesn't mention external validation or publication, so for now this is the company's own account.

The piece grabs attention with stark numbers: LLMs consuming 70% of high-end memory, hard drive prices more than doubling in two years, entry-level PCs possibly gone by 2028. The author's core argument is that this isn't just rapid adoption—it's shockingly inefficient engineering compared to past tech booms like streaming or smartphones. I'd discount that framing a bit. The article cites Gartner forecasts and multiple reports, but never provides direct energy-efficiency comparisons—tokens per watt, architectural differences at equivalent task loads. Without those, "engineering disaster" reads more like a vibe than a technical verdict. The useful bits are the hardware shortage timelines: memory crunch expected to last years, data center capacity planned to grow 8x, companies repurposing jet engines for power. Those are real constraints on the ground, more concrete than the efficiency label.

This fixes a real agent pain point: workflows that die at the login screen. 1Password injects credentials directly into the form, Claude just keeps going — the model never touches the password. Way safer than having the model handle auth itself. The post doesn't say which sites are supported or whether there are rate limits, so I'd treat this as a browser-level patch, not full account delegation yet.

This one's worth opening because the EU just moved DMA enforcement from 'search choice screens' to the OS level. The order says Android must give rival search engines and AI assistants the same access and data that Google's own services get—no more preferential treatment for Gemini or Google Search. The old fights were about default browsers; this one is about whether a third-party AI assistant can get the same click data, voice entry points, and system-level hooks on Android. The post doesn't give a compliance deadline or fine range, so I'd discount the urgency a bit—these orders usually take forever to implement, and Google will almost certainly appeal. But the direction is clear: the EU wants Android to be a level playing field for AI assistants, not Google's walled garden.

The reason to click: the DMA is finally targeting the bundle of AI assistant and search on Android. The core order says third-party AI assistants should get system-level access comparable to Gemini—things like long-pressing the home button or reading screen content. Google also has to share some Search data, but the post doesn't say whether that means click data, query logs, or ranking signals, nor through what API. Google will almost certainly appeal. If past Shopping and Android cases are any guide, real enforcement could take two to three years. For companies building alternative AI assistants or search engines, this is a potential opening—but I'd hold off on counting the revenue until we see the appeal outcome and the actual data-sharing terms.

I clicked because it captures a real moment from this year's tech conferences: speakers criticize LLMs on stage while the audience applauds with Claude Code open. Jeremy Theocharis, CTO at UMH, was at Local-First Conf in Berlin and watched Armin Ronacher (creator of Flask) explain that his project Earendil auto-closes nearly all LLM-generated PRs—while building an open-source coding agent harness. The irony of building the tool that floods you isn't lost on him. He lists the critiques honestly: slop eroding OSS trust, broken junior-senior teaching loops, copyright and geopolitical supply risks. He doesn't try to resolve the tension. He just asks others to share their usage patterns so the community can map this collective unease. I'd read this as a vibe check, not a thesis. No data, no solutions—but the 'I hate it and I use it' posture is closer to how many working devs actually feel than most industry reports admit.

The numbers make this worth a click: 487 days, 300K lines, a 31KB wasm binary. The team is upfront that this was a redesign, not a port, so direct Rust-vs-Zig performance comparisons don't apply. I'd discount the milestone a bit—it's feature parity, not a release; v0.1.0 is planned for later this year. But the wasm size is less than half the original, and hot code loading plus reproducible cross-compilation are real improvements for anyone targeting WebAssembly. Bun's 11-day Zig→Rust post is a useful contrast: that was a direct port, this was a ground-up rebuild, so the 40x time difference isn't surprising.

The reason this catches attention: the donor and the recipient line up almost too neatly. Anthropic's CEO put $1M into a super PAC that pushes AI safety regulation—not a candidate, but the regulatory agenda itself. The problem is the Reddit post body is blocked by network security. All we have is a title and an image. I can't tell if the money is for lobbying a specific bill, running ads, or building a coalition, and there's no word on whether Amodei has commented publicly. Given Anthropic's long-running "responsible scaling" narrative, the donation isn't surprising—but the amount and the vehicle are new. If detailed FEC filings surface, I'd want to trace where the cash actually lands in terms of policy targets.

This one's worth opening because the metaphor is dead-on: throw the dart, then draw the bullseye around it. We assume language flows from intent—first you know what you mean, then you say it. LLMs flip that. Tokens come out one by one, each conditioned on what's already been generated, and a coherent direction only emerges mid-stream. Anthropic's research backs this up: Claude plans rhymes ahead when writing poetry, but if you ask it to explain its choices afterward, the explanation is just another generated continuation—not a faithful readout of internal state. The model's confident self-explanation is itself a new throw. Nick Gustafson frames this lightly, but the implication is sharp. Ask a model five times why it said something, and you might get five plausible-sounding answers. That's a warning for anyone treating model explanations as causal accounts. I'd read this as a concept-calibration piece, not a technical breakthrough—it won't give you a new method, but it'll reset your expectations about what's happening when a model sounds sure of itself.