> ## Documentation Index
> Fetch the complete documentation index at: https://developers.reflection.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# FP8 quantization
> How Beam's FP8 weights were quantized
export const MemoryBars = ({title, desc, series, rows, unit = "GiB", decimals = 2, max}) => {
const width = 720;
const barLeft = 72;
const barRight = 560;
const barHeight = 22;
const barGap = 6;
const top = 92;
const domainMax = max || Math.max(...rows.flatMap(row => row.values));
const scale = value => barLeft + value / domainMax * (barRight - barLeft);
const barPath = (x1, y, h) => {
const r = Math.min(4, (x1 - barLeft) / 2);
return `M ${barLeft} ${y} H ${x1 - r} Q ${x1} ${y} ${x1} ${y + r} V ${y + h - r} Q ${x1} ${y + h} ${x1 - r} ${y + h} H ${barLeft} Z`;
};
const rowBlock = 34 + series.length * (barHeight + barGap) + 14;
const height = top + rows.length * rowBlock;
return ;
};
export const QualityDumbbell = ({title, desc, series, groups, domain = [0, 100], ticks = [0, 20, 40, 60, 80, 100], unit = "%"}) => {
const width = 720;
const plotLeft = 224;
const plotRight = 604;
const changeRight = 704;
const rowHeight = 28;
const groupHeight = 34;
const plotTop = 112;
const scale = value => plotLeft + (value - domain[0]) / (domain[1] - domain[0]) * (plotRight - plotLeft);
const format = value => `${value.toFixed(1)}${unit}`;
const formatChange = (a, b) => {
const change = Math.round((b - a) * 10) / 10;
if (change === 0) return "0.0";
return change > 0 ? `+${change.toFixed(1)}` : `−${Math.abs(change).toFixed(1)}`;
};
let y = plotTop;
const laidOut = groups.map(group => {
const headerY = y + 22;
y += groupHeight;
const rows = group.rows.map(row => {
const rowY = y + rowHeight / 2;
y += rowHeight;
return {
...row,
rowY
};
});
return {
...group,
headerY,
rows
};
});
const plotBottom = y + 6;
const height = plotBottom + 16;
const [nameA, nameB] = series;
return ;
};
[Beam-1.0-501B-A23B-FP8](https://huggingface.co/reflection/Beam-1.0-501B-A23B-FP8) is the FP8-quantized version of [Beam-1.0-501B-A23B](https://huggingface.co/reflection/Beam-1.0-501B-A23B).
## Model details
* **Architecture, capabilities, and intended use:** see [Beam's model card](https://huggingface.co/reflection/Beam-1.0-501B-A23B).
* **License:** Apache 2.0
This checkpoint was quantized using [LLM Compressor](https://github.com/vllm-project/llm-compressor).
## Quantization scheme
The checkpoint uses **FP8\_DYNAMIC (W8A8)** with FP8 E4M3 weights and input activations. Weights have a fixed scale per output channel; activation scales are computed dynamically per token during inference.
Quantization covers the following linear projections:
* **Attention:** query, key, value, output, and attention gate projections.
* **Dense MLPs and MoE experts:** gate, up, and down projections.
Embeddings, the language modeling head, MoE routers, and normalization layers retain their original precision and values.
## Quantization method
We use **post-training quantization (PTQ)** to convert the selected weights of the BF16 Beam checkpoint to FP8 using LLM Compressor's **round-to-nearest (RTN)** method. During conversion, weights are divided by their scales and rounded to the nearest representable FP8 value. This requires no additional training or calibration dataset.
## Evaluation and performance
### Memory footprint
The measurements below were taken on a TP8 endpoint serving on 8×H200 GPUs with a BF16 KV cache.
| **Measurement** | BF16 | FP8 |
| - | - | - |
| Checkpoint size | 933.81 GiB | 468.72 GiB |
| GPU memory used, mean per GPU | 135.08 GiB | 88.40 GiB |
### Quality
All evaluations used temperature 0.7 and top\_p 0.99, with an FP8 KV cache for the FP8 checkpoint.
Two benchmarks report a score rather than a percentage, so they have their own scale:
| | *Beam (BF16)* | *Beam (FP8)* |
| - | - | - |
| **Agentic Coding / Terminal** | | |
| DeepSWE v1.1 | 43.3% | 43.0% |
| SWE Bench Pro v2-Hard | 76.5% | 78.4% |
| SWE Bench Pro v1 | 66.1% | 65.9% |
| Terminal Bench v2.1 | 80.1% | 78.9% |
| SWE Atlas Codebase QnA | 31.8% | 32.2% |
| SWEBench Multilingual | 78.3% | 78.1% |
| SWEBench Verified | 81.6% | 80.2% |
| **Reasoning** | | |
| AIME 2026 | 97.7% | 98.0% |
| HLE (no tools) | 36.0% | 36.0% |
| HLE (with tools) | 51.1% | 51.1% |
| SciCode | 49.2% | 48.3% |
| CriPT AA | 16.3% | 17.4% |
| GPQA Diamond | 90.6% | 90.3% |
| LiveCodeBench | 88.3% | 88.1% |
| **Tool Calling / Search** | | |
| AutomationBench - public split only | 39.3 | 38.2 |
| MCP Atlas | 80.8% | 81.5% |
| Tau3 Banking | 36.1% | 36.8% |
| **General Capabilities** | | |
| AA-LCR | 78.2% | 79.4% |
| LongBench v2 | 67.5% | 67.6% |
| IFBench | 79.6% | 80.7% |
| AA Omniscience (index) - public split only | 10.8 | 8.8 |
## Hardware compatibility and serving
We recommend **CUTLASS for FP8 linear layers** and **Triton for MoE execution** on the following SGLang configurations. GPU counts are per replica and reflect validated layouts, rather than minimum requirements.
| **Hardware** | **Tensor parallelism** | **Linear backend** | **MoE runner** | **Attention backend** |
| - | - | - | - | - |
| 8 NVIDIA H200 GPUs | TP8 | CUTLASS | Triton | FlashAttention 3 (`fa3`) |
| 4 NVIDIA B300 GPUs | TP4 | CUTLASS | Triton | TensorRT-LLM MHA (`trtllm_mha`) |
Select an [SGLang image](https://hub.docker.com/r/lmsysorg/sglang) release with Beam support, and pin it by digest.
```bash icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
docker pull lmsysorg/sglang:beam
```
Launch on **8 H200 GPUs**:
```bash icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
docker run --gpus all \
--ipc=host \
-p 30000:30000 \
-e USE_TRITON_W8A8_FP8_KERNEL=0 \
-e SGLANG_ENABLE_FP8_GEMM_CONFIG_TUNE=0 \
lmsysorg/sglang:beam \
python -m sglang.launch_server \
--model-path reflection/Beam-1.0-501B-A23B-FP8 \
--tensor-parallel-size 8 \
--attention-backend fa3 \
--moe-runner-backend triton \
--kv-cache-dtype fp8_e4m3 \
--host 0.0.0.0 \
--port 30000
```
For **4 B300 GPUs**, use the same command with `--tensor-parallel-size 4` and `--attention-backend trtllm_mha`.
FA4’s FP8 KV cache path on H200 requires a head dimension of 512, while Beam uses a head dimension of 128. We therefore recommend FA3 for this configuration.
See the public [SGLang CLI reference](https://docs.sglang.io/advanced_features/server_arguments.html) for additional options.
Select a [vLLM image](https://docs.vllm.ai/en/latest/deployment/docker/) release with Beam support, and pin it by digest.
```bash icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
docker pull vllm/vllm-openai:reflection-beam
```
Launch on **8 H200 GPUs**:
```bash icon="docker" theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
docker run --gpus all \
--ipc=host \
-p 8000:8000 \
vllm/vllm-openai:reflection-beam \
--model reflection/Beam-1.0-501B-A23B-FP8 \
--tensor-parallel-size 8 \
--attention-backend FLASH_ATTN \
--linear-backend cutlass \
--moe-backend triton \
--kv-cache-dtype fp8_e4m3
```
For **4 B300 GPUs**, use `--tensor-parallel-size 4` and `--attention-backend FLASHINFER`, keeping CUTLASS linears and Triton MoE.
See the public [vLLM CLI reference](https://docs.vllm.ai/en/latest/cli/serve/) for additional options.
We thank the LLM Compressor team for developing and maintaining the quantization tooling used to produce this release and the SGLang team for their inference optimizations. We’re grateful to **Jinze Xue, Matthew Radzihovsky, Zhiyu Cheng, and Trenton Starkey at NVIDIA** for their technical expertise, guidance, and support in developing and refining our quantization recipes.