> ## Documentation Index
> Fetch the complete documentation index at: https://developers.reflection.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# NVFP4 quantization
> How Beam's NVFP4 weights were quantized
export const MemoryBars = ({title, desc, series, rows, unit = "GiB", decimals = 2, max}) => {
const width = 720;
const barLeft = 72;
const barRight = 560;
const barHeight = 22;
const barGap = 6;
const top = 92;
const domainMax = max || Math.max(...rows.flatMap(row => row.values));
const scale = value => barLeft + value / domainMax * (barRight - barLeft);
const barPath = (x1, y, h) => {
const r = Math.min(4, (x1 - barLeft) / 2);
return `M ${barLeft} ${y} H ${x1 - r} Q ${x1} ${y} ${x1} ${y + r} V ${y + h - r} Q ${x1} ${y + h} ${x1 - r} ${y + h} H ${barLeft} Z`;
};
const rowBlock = 34 + series.length * (barHeight + barGap) + 14;
const height = top + rows.length * rowBlock;
return ;
};
[Beam-1.0-501B-A23B-NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) is the NVFP4-quantized version of [Beam-1.0-501B-A23B](https://huggingface.co/reflection/Beam-1.0-501B-A23B). It is about 3.4× smaller than the BF16 release (273 GiB vs. 934 GiB), so a full copy of the model runs on 4 NVIDIA B300 GPUs.
## Model details
* **Architecture, capabilities, and intended use:** see [Beam's model card](https://huggingface.co/reflection/Beam-1.0-501B-A23B).
* **License:** Apache 2.0
This checkpoint was quantized with [NVIDIA Model Optimizer](https://github.com/NVIDIA/Model-Optimizer) and is saved in its format. We thank the NVIDIA team for Model Optimizer and for their technical expertise and support, and the SGLang team for their inference optimizations.
## Quantization scheme
The checkpoint uses **NVFP4 (W4A4)**: weights and input activations are stored as 4-bit E2M1 values. Each block of 16 values shares one FP8 E4M3 scale, and each tensor has one additional global scale.
Quantization covers only the **routed-expert MLPs** (gate, up, and down projections) in layers 3–51. These hold most of the model's parameters, so this is where 4-bit storage saves the most memory.
The dense MLPs in layers 0–2, attention, the MoE routers, the embeddings, the language modeling head, and normalization layers keep their original BF16 precision and values.
## Quantization method
We use **post-training quantization (PTQ)** with Model Optimizer's [**Local Hessian**](https://nvidia.github.io/Model-Optimizer/announcements/local-hessian.html) calibration. For each block, it searches over candidate scales and keeps the one that minimizes the Hessian-weighted error ΔWᵀ H ΔW, where H is computed from the tokens routed to that expert. This requires no additional training.
Calibration uses 1,000 sequences of 32,768 tokens each (32.8M tokens in total) of long-context agentic data.
### KV cache
The checkpoint does not include calibrated KV-cache scales. When serving with `--kv-cache-dtype fp8_e4m3`, the cache is stored in FP8 E4M3 with a scale of 1.0.
### Memory
| | **BF16** | **NVFP4** |
| - | - | - |
| Checkpoint size | 934 GiB | 273 GiB |
## Hardware compatibility and serving
Native NVFP4 requires NVIDIA Blackwell GPUs. We validated the model on the following configuration. The GPU count is per copy of the model and is not a minimum requirement.
| **Hardware** | **Tensor parallelism** | **MoE runner** | **Attention backend** |
| - | - | - | - |
| 4 NVIDIA B300 GPUs | TP4 | FlashInfer TensorRT-LLM (`flashinfer_trtllm`) | TensorRT-LLM MHA (`trtllm_mha`) |
Launch on **4 B300 GPUs**:
```bash icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
python -m sglang.launch_server \
--model-path reflection/Beam-1.0-501B-A23B-NVFP4 \
--tensor-parallel-size 4 \
--attention-backend trtllm_mha \
--moe-runner-backend flashinfer_trtllm \
--reasoning-parser beam \
--tool-call-parser beam \
--kv-cache-dtype fp8_e4m3
```
SGLang detects the quantization format from the checkpoint, so no extra quantization flag is needed. `--kv-cache-dtype fp8_e4m3` stores the attention cache in 8 bits, which fits about twice as many tokens in GPU memory. To keep the cache in full precision, use `--kv-cache-dtype bfloat16` instead.
See the public [SGLang CLI reference](https://docs.sglang.io/advanced_features/server_arguments.html) for additional options.
```bash icon="terminal" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 \
--tensor-parallel-size 4 \
--attention-backend FLASHINFER \
--moe-backend flashinfer_trtllm \
--enable-auto-tool-choice \
--tool-call-parser beam \
--reasoning-parser beam \
--kv-cache-dtype fp8
```
The first start takes longer than later ones because FlashInfer tunes the MoE kernel for the GPU and caches the result.
For **8 H100 GPUs**, use `--tensor-parallel-size 8 --moe-backend marlin` and drop `--attention-backend`. Set the Marlin backend explicitly, because it is faster than the one vLLM selects by default on Hopper.
See the public [vLLM CLI reference](https://docs.vllm.ai/en/latest/cli/serve/) for additional options.
We’re grateful to **Jinze Xue, Matthew Radzihovsky, Zhiyu Cheng, and Trenton Starkey at NVIDIA** for their valuable guidance and support in developing and refining our quantization recipes.