In [_Kimi K3: Open Frontier Intelligence_](https://www.kimi.com/blog/kimi-k3), Moonshot AI explains how it has pushed mixture-of-experts architecture much further.

> “We have also scaled up Mixture of Experts (MoE) sparsity, effectively activating 16 out of 896 experts.”

The concept is not new. Mixtral selects 2 of 8 experts, DBRX selects 4 of 16, DeepSeek-V3 selects 8 of 256 routed experts, and Qwen3-Coder selects 8 of 160. K3 scales that to 896 experts while using only 16 for each token.

What may be unique is the extreme sparsity at 2.8 trillion parameters, along with new routing and balancing techniques designed to keep all of those experts useful.

That is more interesting to me than another benchmark ranking. I want to know whether that many experts develop distinct specialties and how reliably the router can choose the right few for the work.

Read the full [Kimi K3 announcement](https://www.kimi.com/blog/kimi-k3).

In <a href="https://www.kimi.com/blog/kimi-k3">Kimi K3: Open Frontier Intelligence</a>, Moonshot AI explains how it has pushed mixture-of-experts architecture much further.
<blockquote>
“We have also scaled up Mixture of Experts (MoE) sparsity, effectively activating 16 out of 896 experts.”
</blockquote>
The concept is not new. Mixtral selects 2 of 8 experts, DBRX selects 4 of 16, DeepSeek-V3 selects 8 of 256 routed experts, and Qwen3-Coder selects 8 of 160. K3 scales that to 896 experts while using only 16 for each token.
What may be unique is the extreme sparsity at 2.8 trillion parameters, along with new routing and balancing techniques designed to keep all of those experts useful.
That is more interesting to me than another benchmark ranking. I want to know whether that many experts develop distinct specialties and how reliably the router can choose the right few for the work.
Read the full <a href="https://www.kimi.com/blog/kimi-k3">Kimi K3 announcement</a>.

In [_Kimi K3: Open Frontier Intelligence_](https://www.kimi.com/blog/kimi-k3), Moonshot AI explains how it has pushed mixture-of-experts architecture much further.

, Moonshot AI explains how it has pushed mixture-of-experts architecture much further.

“We have also scaled up Mixture of Experts (MoE) sparsity, effectively activating 16 out of 896 experts.”

The concept is not new. Mixtral selects 2 of 8 experts, DBRX selects 4 of 16, DeepSeek-V3 selects 8 of 256 routed experts, and Qwen3-Coder selects 8 of 160. K3 scales that to 896 experts while using only 16 for each token.

What may be unique is the extreme sparsity at 2.8 trillion parameters, along with new routing and balancing techniques designed to keep all of those experts useful.

That is more interesting to me than another benchmark ranking. I want to know whether that many experts develop distinct specialties and how reliably the router can choose the right few for the work.

Read the full [Kimi K3 announcement](https://www.kimi.com/blog/kimi-k3).

Moonshot AI’s Kimi K3 scales mixture-of-experts sparsity to 16 of 896 experts. The interesting question is whether that many experts form distinct specialties and how reliably the router chooses among them.