MoE Memory Requirements: Total vs Active Parameters

A mixture-of-experts model can use a small fraction of its parameters for each token while retaining a much larger set of weights. For a fully GPU-resident deployment, budget for that complete set across the device group. The active parameter count describes selective execution, not the model's storage footprint.

That distinction matters for Korean AI as well as GPU sizing. LG AI Research's K-EXAONE lists 236 billion total parameters and 23 billion activated. If its expert weights stay resident, memory placement and routing efficiency matter alongside the reduced arithmetic. What should a reader check before treating the smaller number as evidence of a cheaper service? Start with the official model card.

First, Check the Bytes Behind the Model Name

Qwen3-30B-A3B provides a manageable example to audit. Its model card lists 30.5 billion total parameters and 3.3 billion activated. Multiplying the active count by two bytes gives 6.6 GB. That arithmetic is correct, but using the result as the full BF16 weight budget is the mistake.

The checkpoint offers a more direct check. Its configuration specifies bfloat16, and its weight index reports a total tensor size of 61,064,245,248 bytes. Those public metadata files were checked on September 14, 2026; no model inference was run.

Divide by one billion for decimal GB: the result is approximately 61.06 GB. Divide by 1,073,741,824 for GiB: approximately 56.87 GiB. These are two units for the same stored tensor payload. The result closely matches the rough total-count estimate of 30.5 billion × two bytes, or 61 GB.

An index is a useful starting point, not a complete serving-memory report. Its tensor total excludes file-format overhead and does not capture how a loader converts, distributes, or duplicates weights. Runtime memory also needs room for the cache and working allocations. The useful finding here is narrower: the official BF16 checkpoint contains roughly 61 GB of tensor data, despite its much smaller active parameter count.

Two Models, the Same Accounting Test

The table applies a uniform BF16 (16-bit) assumption of two bytes per parameter to the rounded counts in the two official model cards. It compares accounting methods, not model quality or measured hardware requirements. All storage values below use decimal GB.

Model	Total / active parameters	BF16: total × 2 bytes	Wrong shortcut: active × 2 bytes
Qwen3-30B-A3B	30.5B / 3.3B	61 GB	6.6 GB
K-EXAONE-236B-A23B	236B / 23B	472 GB	46 GB

These are AI NodeLab calculations: total parameters × two bytes versus the deliberately incorrect active-count shortcut. Qwen's exact index provides the separate cross-check above. The K-EXAONE figure is an approximate calculation from its rounded parameter count, not an independently audited checkpoint size. Neither row specifies a minimum GPU count.

Quantization changes the bytes assigned to stored weights. Sparse routing changes which experts execute for a token. They can be used together, but they solve different parts of the problem. For actual checkpoint reductions and the distinction from cache compression, see weight quantization versus KV cache quantization.

Selected for One Token, Stored for Many Possible Tokens

In these MoE architectures, a router selects among expert feed-forward networks at each MoE layer. Different tokens can select different experts. A fully resident deployment keeps the alternatives available even when a particular token does not use them.

There are also weights outside the routed expert bank. This is why multiplying the entire model's parameter count by the fraction of selected experts gives another misleading shortcut. Qwen lists eight activated experts out of 128; that fraction applies to expert selection, not uniformly to attention, embeddings, and every other parameter.

Illustrative single-layer routing, not a Qwen or K-EXAONE diagram. All eight experts remain stored in this fully resident example; only two are selected for this token. No shared expert is included. Example: AI NodeLab. Open the image to enlarge it.

Batching adds another distinction. Consider a hypothetical single layer with eight equal-sized routed experts, two selected per token, and no shared expert. Four tokens might be routed as follows:

Token in this batch	Selected experts
A	1 and 2
B	2 and 3
C	4 and 5
D	7 and 8

Each token selects two experts, yet the batch touches seven distinct experts through eight token-expert assignments. Under the fully resident assumption, expert 6 remains stored too. This is an illustrative routing pattern, not a trace from Qwen or K-EXAONE.

Keep those three counts separate: eight stored, two selected per token, seven touched by this batch. They describe capacity, selection, and this batch's coverage. They still do not determine execution time: repeated use of an expert and the distribution of work affect how the engine processes the batch.

Putting Experts on Different Devices Changes the Budget

Expert parallelism distributes expert weights across devices. In vLLM's deployment documentation, expert placement and attention-weight placement depend on the parallel configuration. Attention weights may be replicated or sharded, while additional expert replicas used for load balancing consume memory of their own.

Consequently, dividing the full checkpoint size by the number of GPUs is only a preliminary estimate under an even-sharding assumption. A useful per-device budget identifies resident expert weights, other resident weights, KV cache, and runtime or communication buffers. It also records which tensors are duplicated.

Offloading offers a different tradeoff. Hugging Face Accelerate documents layer placement across GPUs, CPU memory, and disk, with transfers for offloaded execution. This allows some weights to live outside GPU memory. It is a specific implementation example; support and movement granularity vary by engine. Ask where the weights live and when they move before comparing its memory figure with an all-resident deployment.

What the Korean Example Adds

K-EXAONE makes this an architectural question for Korea's model developers, not just its memory suppliers. LG's technical explanation describes eight selected routed experts from a pool of 128, alongside one shared expert. The shared expert belongs in the accounting; a diagram showing only eight selected blocks would omit part of the design.

Its model card also specifies a mixture of sliding-window and global attention. Therefore, the active parameter headline cannot answer a second capacity question: how much KV state a service retains. That requires the actual attention layout, cache format, and live token occupancy. The KV cache calculation guide explains which inputs to collect and where the uniform full-attention formula stops applying.

I would judge Korean AI models by more than their active parameter counts. To understand the cost of a real service, I would also ask where the complete weights are stored and how efficiently data reaches the selected experts.

A useful efficiency claim for this model would report the checkpoint and precision, weight placement, workload, task quality, and latency together. Balanced expert execution with little exposed transfer time would support a favorable serving assessment. Heavy transfers or uneven routing that miss the service's latency target would weaken it. Either result would be more informative than the active count alone.

For the next MoE announcement, write down three answers before judging its hardware needs: how many weight bytes are stored, where they reside, and which work the target workload activates. Qwen's index shows how to check the first. The device layout and a measured serving run must supply the other two.