FEATUREDHacker News Frontpage· rssEN08:27 · 09·17
→GLM built its own inference infra on 100k+ Chinese accelerators, tripling throughput in under two weeks
Zhipu AI disclosed how GLM-5.3-Flash inference was built from scratch on a cluster of over 100,000 Chinese-made AI accelerators. The team faced limited chip memory, low bandwidth, and an immature software ecosystem. Instead of relying solely on human engineers, they deployed an Infra Agent powered by GLM-5.3 that turned sparse end-to-end metrics into fine-grained, attributable feedback—kernel-level correctness checks, microbenchmarks, and execution traces—so the agent could pinpoint bottlenecks. Combined with tensor parallelism, W8A8 quantization, mixed-precision KV cache, and an Encode-Prefill-Decode disaggregated architecture, end-to-end throughput improved roughly 3× over the initial baseline, with per-token cost reaching parity with mainstream NVIDIA GPUs. Within a week of launch under the anonymous name Ox-Alpha, the model processed over 62 trillion tokens and became the most-used model on both OpenCode and OpenRouter.
#Code#Agent#Reasoning#Zhipu AI
why featured
Featured · importance 82 · hook + knowledge + resonance
editor take
Zhipu used GLM-5.3 as an Infra Agent to 3× inference throughput on 100k+ domestic chips, reaching cost parity with NVIDIA.
sharp
This one's worth opening because it turns 'running LLMs on domestic chips' from a slogan into a reproducible engineering ledger. Zhipu didn't throw engineers at the problem—they had GLM-5.3 read kernel correctness checks, microbenchmarks, and execution traces to pinpoint bottlenecks directly. Combined with tensor parallelism, W8A8 quantization, mixed-precision KV cache, and an EPD disaggregated architecture, end-to-end throughput roughly tripled, with per-token cost reaching parity with mainstream NVIDIA GPUs. Launched anonymously as Ox-Alpha, it processed 62 trillion tokens in a week and became the most-used model on both OpenCode and OpenRouter.
I'd discount this a bit: the post doesn't name the specific chip model or per-chip memory, and the 'cost parity' baseline isn't spelled out. But the Infra Agent's feedback loop is the real thing—breaking sparse end-to-end metrics into fine-grained, attributable signals goes beyond 'AI writes code' and into AI-assisted system tuning. If this methodology transfers to other domestic chip clusters, that matters more than a single performance number.
HKR breakdown
hook ✓knowledge ✓resonance ✓