NVFP4 quantization

Beam-1.0-501B-A23B-NVFP4 is the NVFP4-quantized version of Beam-1.0-501B-A23B. It is about 3.4× smaller than the BF16 release (273 GiB vs. 934 GiB), so a full copy of the model runs on 4 NVIDIA B300 GPUs.

Model details

Architecture, capabilities, and intended use: see Beam’s model card.
License: Apache 2.0

This checkpoint was quantized with NVIDIA Model Optimizer and is saved in its format. We thank the NVIDIA team for Model Optimizer and for their technical expertise and support, and the SGLang team for their inference optimizations.

Quantization scheme

The checkpoint uses NVFP4 (W4A4): weights and input activations are stored as 4-bit E2M1 values. Each block of 16 values shares one FP8 E4M3 scale, and each tensor has one additional global scale. Quantization covers only the routed-expert MLPs (gate, up, and down projections) in layers 3–51. These hold most of the model’s parameters, so this is where 4-bit storage saves the most memory. The dense MLPs in layers 0–2, attention, the MoE routers, the embeddings, the language modeling head, and normalization layers keep their original BF16 precision and values.

Quantization method

We use post-training quantization (PTQ) with Model Optimizer’s Local Hessian calibration. For each block, it searches over candidate scales and keeps the one that minimizes the Hessian-weighted error ΔWᵀ H ΔW, where H is computed from the tokens routed to that expert. This requires no additional training. Calibration uses 1,000 sequences of 32,768 tokens each (32.8M tokens in total) of long-context agentic data.

KV cache

The checkpoint does not include calibrated KV-cache scales. When serving with --kv-cache-dtype fp8_e4m3, the cache is stored in FP8 E4M3 with a scale of 1.0.

Memory

Exact numbers

	BF16	NVFP4
Checkpoint size	934 GiB	273 GiB

Hardware compatibility and serving

Native NVFP4 requires NVIDIA Blackwell GPUs. We validated the model on the following configuration. The GPU count is per copy of the model and is not a minimum requirement.

Hardware	Tensor parallelism	MoE runner	Attention backend
4 NVIDIA B300 GPUs	TP4	FlashInfer TensorRT-LLM (`flashinfer_trtllm`)	TensorRT-LLM MHA (`trtllm_mha`)

SGLang
vLLM

Launch on 4 B300 GPUs:

python -m sglang.launch_server \
  --model-path reflection/Beam-1.0-501B-A23B-NVFP4 \
  --tensor-parallel-size 4 \
  --attention-backend trtllm_mha \
  --moe-runner-backend flashinfer_trtllm \
  --reasoning-parser beam \
  --tool-call-parser beam \
  --kv-cache-dtype fp8_e4m3

SGLang detects the quantization format from the checkpoint, so no extra quantization flag is needed. --kv-cache-dtype fp8_e4m3 stores the attention cache in 8 bits, which fits about twice as many tokens in GPU memory. To keep the cache in full precision, use --kv-cache-dtype bfloat16 instead.See the public SGLang CLI reference for additional options.

vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 \
  --tensor-parallel-size 4 \
  --attention-backend FLASHINFER \
  --moe-backend flashinfer_trtllm \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam \
  --kv-cache-dtype fp8

The first start takes longer than later ones because FlashInfer tunes the MoE kernel for the GPU and caches the result.For 8 H100 GPUs, use --tensor-parallel-size 8 --moe-backend marlin and drop --attention-backend. Set the Marlin backend explicitly, because it is faster than the one vLLM selects by default on Hopper.See the public vLLM CLI reference for additional options.

We’re grateful to Jinze Xue, Matthew Radzihovsky, Zhiyu Cheng, and Trenton Starkey at NVIDIA for their valuable guidance and support in developing and refining our quantization recipes.

Get started

Self-hosting

Third-party providers

Tools

Model details

Quantization scheme

Quantization method

KV cache

Memory

Hardware compatibility and serving

​Model details

​Quantization scheme

​Quantization method

​KV cache

​Memory

​Hardware compatibility and serving

Model details

Quantization scheme

Quantization method

KV cache

Memory

Hardware compatibility and serving