FEATUREDAI HOT (Curated Pool)· aihot-apiZH23:41 · 07·12
→Ploy migrated its production AI agent from Claude Opus 4.8 to GPT-5.6 Sol: 2.2× faster builds, 27% cheaper
Ploy's agent builds real marketing websites. Claude Opus 4.8 held the default slot for four months with no challenger. GPT-5.6 Sol changed that on launch day. Head-to-head: mean build time dropped from 8m to 3m 42s, cost from $3.06 to $2.22, visual score rose from 0.936 to 0.970. The migration wasn't plug-and-play. Their eval harness was tuned to Opus's sequential style; GPT-5.6's parallel tool calls blew through budgets, and roughly a third of initial failures were harness assumptions, not model errors. GPT-5.6 also writes leaner code—one case went from 17,957 CSS characters and 174 variables to 2,508 characters and 45 variables. Design output is clean and modern but tends toward uniformity; brand adherence required extra steering. The post flags tool schemas, caching, and reasoning replay as additional migration steps but doesn't detail the fixes.
#Agent#Code#Benchmarking#Ploy
why featured
Featured · importance 82 · hook + knowledge + resonance
editor take
Ploy swapped its production website-building agent from Claude Opus 4.8 to GPT-5.6 Sol: build time halved, cost down 27%, visual score up.
sharp
This post is worth reading because Ploy's agent isn't a demo—it builds real marketing websites in production. Claude Opus 4.8 held that slot for four months with no challenger. GPT-5.6 Sol got tested on launch day, and the numbers are stark: mean build time dropped from 8 minutes to 3m 42s, cost from $3.06 to $2.22, visual score from 0.936 to 0.970.
The migration wasn't plug-and-play. Their eval harness was tuned to Opus's sequential tool-calling style; GPT-5.6's parallel calls blew through budgets on cases it was solving correctly. Roughly a third of initial failures were harness assumptions, not model errors. GPT-5.6 also writes leaner code—one case went from 17,957 CSS characters and 174 variables to 2,508 characters and 45 variables. The tradeoff: design output trends uniform, and brand adherence needed extra steering.
The post flags tool schemas, caching, and reasoning replay as additional migration steps but doesn't detail the fixes. I'd treat this as a practical field manual for production model swaps, which is more useful than any benchmark leaderboard.
HKR breakdown
hook ✓knowledge ✓resonance ✓