Tencent open-sources Hy4 preview under Apache 2.0: 770B parameters, 49B active

Tencent released Hy4 preview as an open-weights model on August 28, 2026, distributing the weights under the Apache 2.0 license. Hy4 preview is a mixture-of-experts model that holds 770 billion total parameters but activates only 49 billion per token, with a context window exceeding 1 million tokens. In an internal evaluation Tencent ran with 163 experts across 203 engineering tasks, the model scored 2.99 out of 4.00, ahead of GLM-5.3 at 2.92 and Kimi K3 at 2.94. API pricing is $0.834 per million input tokens, $2.501 per million output tokens, and $0.042 per million cache-hit tokens. ASAP breaks down the architecture and the numbers, based on Tencent's announcement and the Hugging Face model card.

Only 49 billion of the 770 billion switch on

The defining design choice in Hy4 preview is the gap between how large the model is and how much of it runs. Total parameters reach 770 billion, but only 49 billion activate for any single token. The stack is 78 layers deep, each layer holding 256 routed experts plus 1 shared expert, with the top 8 routed experts selected per token. Hidden size is 6,144 and the vocabulary is 120,832. The model card specifies Gated DeepSeek Sparse Attention with an IndexCache. A 1 million token window rules out dense attention on cost grounds, so the sparse attention path and its cache are the structures actually carrying the long context.

The published scores cluster around coding and reasoning

Every benchmark the model card publishes sits on the coding and scientific reasoning axis. The headline figures are GPQA Diamond at 92.3%, SWE-bench Multilingual at 82.9%, SWE Bench Pro at 65.7%, and Deep SWE at 64.3%. Tencent says the model was co-designed with internal experts in software engineering, office analysis, game development, and scientific research. Weights are hosted on Hugging Face, ModelScope, GitCode, and CNB, while product access runs through WorkBuddy, CodeBuddy, Yuanbao, and ima, with API access through Tencent Cloud TokenHub and OpenRouter.

The free window has a hard edge

Tencent opened free access on WorkBuddy and CodeBuddy for two weeks at launch. Free use of the previous generation, Hy3, was extended through September 30. Paid pricing lands at $0.834 per million input tokens and $2.501 per million output tokens, with cache hits at $0.042 per million. The line that matters most in practice is that last one, because a cache hit costs about 5% of a fresh input token. For long-context work that reloads the same documents on every turn, the real invoice diverges sharply from the headline rate.

Reading a benchmark the company scored itself

The most interesting editorial choice here is that Tencent led with an internal expert panel rather than a public benchmark. A result produced by 163 experts grading 203 engineering tasks on a 4-point scale cannot be reproduced by anyone outside the company, and neither the task mix nor the rubric was published. The reasoning behind such a choice is not hard to reconstruct. Public benchmarks are saturated at the top, contamination disputes follow them everywhere, and scores routinely diverge from what practitioners feel when they use a model. Human grading aims squarely at that gap. The catch is that when the grader and the builder are the same company, the metric becomes indistinguishable from marketing material. Publishing reproducible SWE-bench numbers alongside it is what keeps the release from resting on the panel alone.

Why 0.05 points is not a ranking

Looking at 2.99, 2.94, and 2.92 together shows an evaluation that is closer to a tie than to a leaderboard. The margin over Kimi K3 is 0.05 points and the margin over GLM-5.3 is 0.07, on a 4-point scale, with no standard deviations or confidence intervals disclosed. Nothing in the announcement establishes that those gaps clear the noise floor. The table still says something worth hearing. The top three open-weights models out of China now land inside a band that human graders cannot separate on engineering work. The practical reading is that the deciding factors have moved off raw capability and onto price, license, and how hard the thing is to serve.

Can a team actually run 770 billion parameters in house

The number that blocks self-hosting is 770 billion, not the 49 billion that activate. Mixture-of-experts routing cuts computation per token, but the full weight set still has to sit in memory somewhere. At one byte per parameter under 8-bit quantization, the arithmetic alone puts that near 770 gigabytes; at half a byte under 4-bit, near 385 gigabytes. A 1 million token context then adds a KV cache on top of that. Weights being open under Apache 2.0 and weights being servable inside your own infrastructure are entirely separate questions. For most teams the realistic entry point is the TokenHub or OpenRouter API, with self-hosting reserved for organizations that can stand up a multi-node deployment. Apache 2.0 still matters for a different reason: it removes the license review step from shipping a fine-tuned derivative commercially.

Take the word preview literally

The word preview in the model name is doing real work. Tencent shipped this as a preview rather than a general-availability release, and the announcement fixes neither a date for the final version nor a guarantee that its scores will match these. The 2.99 panel result is externally unverifiable, and the published benchmark figures are Tencent's own measurements. A 1 million token window describes the size of the context, not sustained quality across it, and no long-context effectiveness result was included in this release. Any team evaluating adoption should keep the step where they re-measure on their own tasks.

Source: Tencent, "Tencent Releases and Open-Sources Tencent Hy4 preview" (August 28, 2026; 770B total and 49B active parameters, context window exceeding 1M tokens, internal evaluation with 163 experts across 203 engineering tasks scoring 2.99/4.00 against GLM-5.3 at 2.92 and Kimi K3 at 2.94, pricing of $0.834 per million input tokens, $2.501 output and $0.042 cache hits, two weeks of free access on WorkBuddy and CodeBuddy with Hy3 free access extended through September 30, availability via Tencent Cloud TokenHub and OpenRouter) and the Hugging Face tencent/Hy4-preview model card (Apache 2.0 license, 78 layers with 256 routed experts and 1 shared expert with top-8 routing, hidden size 6,144 and vocabulary 120,832, Gated DeepSeek Sparse Attention with IndexCache, GPQA Diamond 92.3%, SWE-bench Multilingual 82.9%, SWE Bench Pro 65.7%, Deep SWE 64.3%, distribution on Hugging Face, ModelScope, GitCode and CNB), compiled by ASAP.