FP8 quantization

Beam-1.0-501B-A23B-FP8 is the FP8-quantized version of Beam-1.0-501B-A23B.

Model details

Architecture, capabilities, and intended use: see Beam’s model card.
License: Apache 2.0

This checkpoint was quantized using LLM Compressor.

Quantization scheme

The checkpoint uses FP8_DYNAMIC (W8A8) with FP8 E4M3 weights and input activations. Weights have a fixed scale per output channel; activation scales are computed dynamically per token during inference. Quantization covers the following linear projections:

Attention: query, key, value, output, and attention gate projections.
Dense MLPs and MoE experts: gate, up, and down projections.

Embeddings, the language modeling head, MoE routers, and normalization layers retain their original precision and values.

Quantization method

We use post-training quantization (PTQ) to convert the selected weights of the BF16 Beam checkpoint to FP8 using LLM Compressor’s round-to-nearest (RTN) method. During conversion, weights are divided by their scales and rounded to the nearest representable FP8 value. This requires no additional training or calibration dataset.

Evaluation and performance

Memory footprint

The measurements below were taken on a TP8 endpoint serving on 8×H200 GPUs with a BF16 KV cache.

Exact numbers

Measurement	BF16	FP8
Checkpoint size	933.81 GiB	468.72 GiB
GPU memory used, mean per GPU	135.08 GiB	88.40 GiB

Quality

All evaluations used temperature 0.7 and top_p 0.99, with an FP8 KV cache for the FP8 checkpoint. Two benchmarks report a score rather than a percentage, so they have their own scale:

Exact numbers

	Beam (BF16)	Beam (FP8)
Agentic Coding / Terminal
DeepSWE v1.1	43.3%	43.0%
SWE Bench Pro v2-Hard	76.5%	78.4%
SWE Bench Pro v1	66.1%	65.9%
Terminal Bench v2.1	80.1%	78.9%
SWE Atlas Codebase QnA	31.8%	32.2%
SWEBench Multilingual	78.3%	78.1%
SWEBench Verified	81.6%	80.2%
Reasoning
AIME 2026	97.7%	98.0%
HLE (no tools)	36.0%	36.0%
HLE (with tools)	51.1%	51.1%
SciCode	49.2%	48.3%
CriPT AA	16.3%	17.4%
GPQA Diamond	90.6%	90.3%
LiveCodeBench	88.3%	88.1%
Tool Calling / Search
AutomationBench - public split only	39.3	38.2
MCP Atlas	80.8%	81.5%
Tau3 Banking	36.1%	36.8%
General Capabilities
AA-LCR	78.2%	79.4%
LongBench v2	67.5%	67.6%
IFBench	79.6%	80.7%
AA Omniscience (index) - public split only	10.8	8.8

Hardware compatibility and serving

We recommend CUTLASS for FP8 linear layers and Triton for MoE execution on the following SGLang configurations. GPU counts are per replica and reflect validated layouts, rather than minimum requirements.

Hardware	Tensor parallelism	Linear backend	MoE runner	Attention backend
8 NVIDIA H200 GPUs	TP8	CUTLASS	Triton	FlashAttention 3 (`fa3`)
4 NVIDIA B300 GPUs	TP4	CUTLASS	Triton	TensorRT-LLM MHA (`trtllm_mha`)

SGLang
vLLM

Select an SGLang image release with Beam support, and pin it by digest.

docker pull lmsysorg/sglang:beam

Launch on 8 H200 GPUs:

docker run --gpus all \
  --ipc=host \
  -p 30000:30000 \
  -e USE_TRITON_W8A8_FP8_KERNEL=0 \
  -e SGLANG_ENABLE_FP8_GEMM_CONFIG_TUNE=0 \
  lmsysorg/sglang:beam \
  python -m sglang.launch_server \
  --model-path reflection/Beam-1.0-501B-A23B-FP8 \
  --tensor-parallel-size 8 \
  --attention-backend fa3 \
  --moe-runner-backend triton \
  --kv-cache-dtype fp8_e4m3 \
  --host 0.0.0.0 \
  --port 30000

For 4 B300 GPUs, use the same command with --tensor-parallel-size 4 and --attention-backend trtllm_mha.

FA4’s FP8 KV cache path on H200 requires a head dimension of 512, while Beam uses a head dimension of 128. We therefore recommend FA3 for this configuration.

See the public SGLang CLI reference for additional options.

Select a vLLM image release with Beam support, and pin it by digest.

docker pull vllm/vllm-openai:reflection-beam

Launch on 8 H200 GPUs:

docker run --gpus all \
  --ipc=host \
  -p 8000:8000 \
  vllm/vllm-openai:reflection-beam \
  --model reflection/Beam-1.0-501B-A23B-FP8 \
  --tensor-parallel-size 8 \
  --attention-backend FLASH_ATTN \
  --linear-backend cutlass \
  --moe-backend triton \
  --kv-cache-dtype fp8_e4m3

For 4 B300 GPUs, use --tensor-parallel-size 4 and --attention-backend FLASHINFER, keeping CUTLASS linears and Triton MoE.See the public vLLM CLI reference for additional options.

We thank the LLM Compressor team for developing and maintaining the quantization tooling used to produce this release and the SGLang team for their inference optimizations. We’re grateful to Jinze Xue, Matthew Radzihovsky, Zhiyu Cheng, and Trenton Starkey at NVIDIA for their technical expertise, guidance, and support in developing and refining our quantization recipes.

Get started

Self-hosting

Third-party providers

Tools

Model details

Quantization scheme

Quantization method

Evaluation and performance

Memory footprint

Quality

Hardware compatibility and serving

​Model details

​Quantization scheme

​Quantization method

​Evaluation and performance

​Memory footprint

​Quality

​Hardware compatibility and serving

Model details

Quantization scheme

Quantization method

Evaluation and performance

Memory footprint

Quality

Hardware compatibility and serving